Prediction engine for a network-based service

ABSTRACT

Service providers can be identified to fulfill service requests of a network-based service. A network system is configured to generate, based on historical data associated with the network-based service, a machine-learned service provider optimization (MLSPO) model for generating service provider optimizations. The optimizations can include action recommendations that optimize one or more service metrics. The MLSPO model can be a reinforcement learning model generated by performing a plurality of simulations utilizing one or more virtual agents. A provider device of a service provider can transmit a set of data to the network system that indicates a current location of the service provider. Based on the current location and the MLSPO model, the network system can generate service provider optimizations. Optimization data can be transmitted to the provider device so that the provider device can display information corresponding to the service provider optimizations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit of priority to U.S. Provisional PatentApplication No. 62/749,413, filed on Oct. 23, 2018, and titled“Prediction Engine”; the aforementioned application being herebyincorporated by reference in its entirety.

BACKGROUND

A network-based service can enable users to request and receive variousnetwork-based services through applications on mobile computing devices.The network-based service can match a service provider with a requestinguser based on the current location of the service provider and a startlocation specified by the requesting user or determined based on thecurrent location of the requesting user.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure herein is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings in which likereference numerals refer to similar elements, and in which:

FIG. 1 is a block diagram illustrating an example network system incommunication with user devices and provider devices, in accordance withexamples described herein;

FIG. 2 is a flow chart illustrating an example method of performingservice provider optimizations, in accordance with examples describedherein;

FIG. 3A is a flow chart illustrating an example method of simulating aninstance of the network-based service as part of the process forgenerating a machine-learned service provider optimization model, inaccordance with examples described herein;

FIG. 3B is a diagram illustrating an exemplary visualization asimulation of an instance of the network-based service as part of theprocess for generating a machine-learned service provider optimizationmodel, in accordance with examples described herein;

FIG. 4 is a diagram illustrating an exemplary visualization of serviceprovider optimizations provided by an exemplary machine-learned serviceprovider optimization model, in accordance with examples describedherein;

FIG. 5 is a block diagram illustrating an example mobile computingdevice, in accordance with examples described herein; and

FIG. 6 is a block diagram that illustrates a computer system upon whichexamples described herein may be implemented.

DETAILED DESCRIPTION

A network system is provided herein that manages a network-based service(e.g., a transport service, a delivery service, etc.) linking availableservice providers (e.g., drivers and/or autonomous vehicles (AVs)) withrequesting users (e.g., riders, service requesters) throughout a givenregion (e.g., San Francisco Bay Area). In doing so, the network systemcan receive requests for service from requesting users via a designateduser application executing on the users' mobile computing devices (“userdevices”). Based on a start location (e.g., a pick-up location where aservice provider is to rendezvous with the requesting user), the networksystem can identify an available service provider and transmit aninvitation to a mobile computing device of the identified serviceprovider (“provider device”). Should the identified service provideraccept the invitation, the network system can transmit directions to theprovider device to enable the service provider to navigate to the startlocation and subsequently from the start location to a service location(e.g., a drop-off location where the service provider is to complete therequested service). The start location can be specified in the requestand can be determined from a user input or from one or more geo-awareresources on the user device. The service location can also be specifiedin the request.

In determining an optimal service provider to fulfill a given servicerequest, the network system can identify a plurality of candidateservice providers to service the service request based on a startlocation indicated in the service request. For example, the networksystem can determine a geo-fence surrounding the start location (or ageo-fence defined by a radius away from the start location), identify aset of candidate service providers (e.g., twenty or thirty serviceproviders within the geo-fence), and select an optimal service provider(e.g., closest service provider to the start location, service providerwith the shortest estimated travel time from the start location, serviceprovider traveling or en-route to a location within a specified distanceor specified travel time to a service location, etc.) from the candidateservice providers to service the service request. In many examples, theservice providers can either accept or decline the invitation based on,for example, the route being too long or impractical for the serviceprovider. After accepting the invitation, the selected service providercan proceed to the start location to, for example, rendezvous with therequesting user. The selected service provider can then proceed inproviding the requested service to the service location. While theservice provider is operating the vehicle to the start location or tothe service location, the provider device can generate directions (e.g.,turn-by-turn navigation directions) to aid the service provider innavigating to the various locations. The directions can be displayed viathe provider application or via a third-party mapping or navigationapplication.

After the service provider completes the requested service, and prior toaccepting another invitation to fulfill anther service request, theservice provider can be described as being on an off-service segment.During this time period, the service provider can take various actions.As one example, the service provider can navigate to another location.As another example, the service provider can remain at the same location(e.g., service location of the previous service request) while waitingfor the next invitation. Service provider can also enter an offlinestate with respect to the network-based service. As used herein, theoffline state can be used to mean a state in which the service provideris not available to fulfill service requests. The service provider canelect to enter the offline state via the provider application. In somevariations, the offline state can be implemented as a server-sidefunction. In other words, the network system can decide to not transmitany invitations to the provider device while the service provider is inthe offline state. In other variations, the offline state can beimplemented as a client-side function where the provider applicationautomatically rejects any invitations received while the serviceprovider is in the offline state.

While on the off-service segment, the service provider may wish to takeparticular actions to improve one or more aspects of the network-basedservice for himself or herself. For example, the service provider maywish to increase or maximize his or her expected fares received fromfulfilling service requests of the network-based service. In addition toor as an alternative, the service provider may wish to decrease orminimize the length of time of or distance traveled while on theoff-service segment (e.g., minimizing expected time and/or distancetraveled to the start location of the next service request).Furthermore, the service provider may wish to decrease or minimize thedistance he or she would have to travel to rendezvous with requestingusers.

In various implementations, the embodiments described herein providedfor a network system that can generate optimizations for serviceproviders to improve one or more service metrics (e.g., increaseexpected earnings, reduction in travel times, reduction in traveldistances, etc.) associated with the network-based service. At ahigh-level, the network system can generate, based on historical dataassociated with the network-based service, one or more machine-learnedservice provider optimization (MLSPO) models for generating serviceprovider optimizations. As used herein, service provider optimizationscan include routing directions (e.g., navigation instructions) and/orsuggestions for performing actions (e.g., entering and exiting offlinemode, etc.) to optimize (e.g., maximize or minimize) one or more servicemetrics for the service provider in fulfilling future service requests.In one example, the network system can generate MLSPO models thatgenerate service provider optimizations that increase or maximizeexpected fares of the service provider. In another example, the networksystem can generate MLSPO models that output service provideroptimizations that decrease or minimize distances traveled by theservice provider in-between service requests (e.g., from previousservice location to subsequent start location).

In one aspect, embodiments described herein can predict, based onhistorical service data, one or more expected service metrics forservice providers given various service provider routes and/or actionsand can identify for the service provider routes and/or actions thatwould optimize the one or more service metrics. The predictions can beresults of machine-learning, such as deep reinforcement learning, inwhich agent-based simulations are performed using the historical servicedata.

As used herein, the terms “optimize,” “optimization,” “optimizing,” andthe like are not intended to be restricted or limited to processes thatachieve the most optimal outcomes. Rather, these terms encompasstechnological processes (e.g., heuristics, stochastics modeling, machinelearning, reinforced learning, Monte Carlo methods, Markov decisionprocesses, etc.) that aim to achieve desirable results. Similarly, termssuch as “minimize” and “maximize” are not intended to be restricted orlimited to processes or results that achieve the absolute minimum orabsolute maximum possible values of a metric, parameter, or variable.

According to embodiments, an input to the MLSPO models is the currentlocation of the service provider. The network system can receivelocation data generated by the provider device that indicates thecurrent location of the service provider. Based on the current locationof the service provider and the MLSPO model, the network system cangenerate optimizations for the service provider. Optimization data canbe transmitted by the network system to the provider device. In responseto receiving the optimization data, the provider device can displayinformation regarding the service provider optimizations to the serviceprovider.

According to embodiments, the service provider optimizations cancorrespond to one or more action recommendations for the serviceprovider and can include a recommended direction of travel for theservice provider while the service provider is traveling on anoff-service segment. The recommended direction of travel can be adirection of travel and/or a route of travel for the service provider tofollow on the off-service segment that would, based on the MLSPO model,achieve optimal expected values of one or more service metrics (e.g.,for one or more future fulfillments of service requests) given theservice provider's current location. In some implementations, the routedirection generated by the MLSPO model can be a direction of travel oran angle of travel from a reference direction (e.g., North). In responseto receiving the optimization data, the provider device can beconfigured to display information to the service provider based on thedirection of travel. For instance, the provider device can display therecommendations to the service provider to travel in a general directionor towards a nearby landmark. In another aspect, the angle of travelfrom the reference direction can be converted to a route direction(e.g., a navigation route along streets and throughways that isdetermined to align with the angle of travel) such that the providerdevice can display a navigation route to the service provider to followthe recommendation. The route direction can be generated based on theangle of travel and the service provider's current location. Accordingto variations, the service provider optimizations can further includeaction suggestions. For example, an action suggestion for the serviceprovider can be for the service provider to enter or exit the offlinestate during the in-between service requests time period.

Depending on the implementation, the network system can generate serviceprovider optimizations in response to data transmitted from the providerdevice (or automatically based on monitoring the location and/or serviceprogress of the service provider). In one variation, the provider devicecan transmit an optimization request to the network system to triggerthe generation of service provider optimizations. The optimizationrequest can be transmitted by the provider device in response to theservice provider interacting with the provider application (e.g., via adedicated soft feature selection of within the provider application). Insome instances, the network system can generate service provideroptimizations in response to receiving an indication that the serviceprovider has completed providing requested services for a requestinguser or the network system can interpret a service completion indicationreceived from the provider device as the optimization request. In othervariations, the network system can automatically generate theoptimizations for the service provider by monitoring the location of theservice provider or the progress of a requested service. As one example,the network system can communicate with the provider device (e.g., byperiodically receiving location data) to monitor the location of theservice provider. And in response to determining that the providerdevice is within a predetermined distance of the service location, thenetwork system can be triggered to begin the process of determiningservice provider optimizations for the service provider. As anotherexample, in response to receiving a service provider's acceptance of aninvitation to fulfill a service request to a service location, thenetwork system can begin to generate optimizations for the serviceprovider based on the machine-learned models and the service location(and/or the estimated arrival time at the service location). In thesemanners, the optimizations can be readily transmitted to the providerdevice when the service provider completes the requested service at theservice location.

According to embodiments, various service metrics associated with thenetwork-based service can be optimized as described herein. Anddifferent MLSPO models can be generated for optimizing estimated orexpected service metrics. In one example, one or more MLSPO models canbe generated to optimize (e.g., maximize) fare(s) the service providercan expect to receive over a given time period in exchange forfulfilling service requests. In another example, other MLSPO models canbe generated to optimize (e.g., minimize) distances the service providercan expect to travel.

As can be appreciated, the machine-learned models to generate serviceprovider optimizations can be trained by a number of differentmachine-learning techniques. In certain implementations, reinforcementlearning can be utilized and a plurality of simulations of a virtualagent in fulfilling the network-based service can be performed usinghistorical data to generate the MLSPO model. Various parameters andmetrics are computed during the simulation by the network system basedon historical data associated with the network system. The parametersand metrics are recorded and used to generate, train, and/or refine themachine-learned model. One or more reinforcement learning techniques,such as Q-learning, can be applied to generate one or moremachine-learned models or policies based on the parameters and metricscomputed during the simulations of the simulations of the network-basedservice.

In certain situations, such as when a large number of service providersare expected in the geographic region or a sub-region therein, theapproach of utilizing single-agent simulations can have drawbacks. Forexample, if a large number of service providers are following the sameset of service provider optimizations, the result for each serviceprovider can be skewed or rendered suboptimal by other service providersfollowing the same set of optimizations. To better optimize for multipleservice providers, the network system can generate multi-agent MLSPOmodels. The multi-agent MLSPO models can be generated by performingmulti-agent simulations. In such simulations, multiple virtual agentsare used in simulating the network-based service.

According to embodiments, the network system can further determine oneor more real-time parameters in generating service provideroptimization. The real-time parameters can reflect real-time conditionsof the service provider at the time the optimization is requested.Real-time parameters can include, for example, real-time demand of thenetwork-based service in various locations in the geographic regionmanaged by the network system, number of other service providers nearby,etc. In some implementations, the real-time parameters can be used asadditional inputs to the MLSPO models to generate the service provideroptimizations.

Compared to existing approaches, embodiments described herein providefor an improved and more efficient way to programmatically generateservice provider optimizations. In one aspect, the network system canmore efficiently utilize processing resources such thatcomputationally-intensive aspects of generating the service provideroptimizations, such as performing agent-based simulations and generatingthe MLSPO models, need not be performed in real-time when optimizationsare desired by the service providers. Accordingly, such computations canbe performed when the demand on the network system is low (e.g.,overnight or during periods of low demand for the network-based servicewhen the network system has unused processing resources). And whenoptimizations are requested, the network system can retrieve the MLPSOmodels from one or more databases and utilize the MLSPO models togenerate the service provider optimizations without needing tore-perform those computationally-intensive steps of the process inreal-time.

In another aspect, the network system can generate more optimal routesand action suggestions. Existing approaches to generate recommendationsfor service providers are simplistic approaches such as directing aservice provider to nearby locations with the highest demand for thenetwork-based service. Such recommendations are often biased towardsshort-term gain (e.g., for only the next service request fulfilled). Incontrast, embodiments described herein generated MLPSO models based onagent-based simulations that simulate service provider actions over timeperiods of many hours and can also factor in other parameters such aswaiting times and distance traveled. Additionally, a shortcoming ofexisting approaches is that when recommendations are followed by manyservice providers, benefits to the service providers can be greatlydiminished. In contrast, embodiments described herein provide formulti-agent MLSPO models that can generate optimizations for a pluralityof service providers.

As used herein, a computing device refers to devices corresponding todesktop computers, cellular devices or smartphones, personal digitalassistants (PDAs), laptop computers, virtual reality (VR) or augmentedreality (AR) headsets, tablet devices, television (IP Television), etc.,that can provide network connectivity and processing resources forcommunicating with the system over a network. A computing device canalso correspond to custom hardware, in-vehicle devices, or on-boardcomputers, etc. The computing device can also operate a designatedapplication configured to communicate with the network service.

One or more examples described herein provide that methods, techniques,and actions performed by a computing device are performedprogrammatically, or as a computer-implemented method. Programmatically,as used herein, means through the use of code or computer-executableinstructions. These instructions can be stored in one or more memoryresources of the computing device. A programmatically performed step mayor may not be automatic.

One or more examples described herein can be implemented usingprogrammatic modules, engines, or components. A programmatic module,engine, or component can include a program, a sub-routine, a portion ofa program, or a software component or a hardware component capable ofperforming one or more stated tasks or functions. As used herein, amodule or component can exist on a hardware component independently ofother modules or components. Alternatively, a module or component can bea shared element or process of other modules, programs or machines.

Some examples described herein can generally require the use ofcomputing devices, including processing and memory resources. Forexample, one or more examples described herein may be implemented, inwhole or in part, on computing devices such as servers, desktopcomputers, cellular or smartphones, personal digital assistants (e.g.,PDAs), laptop computers, VR or AR devices, printers, digital pictureframes, network equipment (e.g., routers) and tablet devices. Memory,processing, and network resources may all be used in connection with theestablishment, use, or performance of any example described herein(including with the performance of any method or with the implementationof any system).

Furthermore, one or more examples described herein may be implementedthrough the use of instructions that are executable by one or moreprocessors. These instructions may be carried on a computer-readablemedium. Machines shown or described with figures below provide examplesof processing resources and computer-readable mediums on whichinstructions for implementing examples disclosed herein can be carriedand/or executed. In particular, the numerous machines shown withexamples of the invention include processors and various forms of memoryfor holding data and instructions. Examples of computer-readable mediumsinclude permanent memory storage devices, such as hard drives onpersonal computers or servers. Other examples of computer storagemediums include portable storage units, such as CD or DVD units, flashmemory (such as carried on smartphones, multifunctional devices ortablets), and magnetic memory. Computers, terminals, network enableddevices (e.g., mobile devices, such as cell phones) are all examples ofmachines and devices that utilize processors, memory, and instructionsstored on computer-readable mediums. Additionally, examples may beimplemented in the form of computer-programs, or a computer usablecarrier medium capable of carrying such a program.

System Descriptions

FIG. 1 is a block diagram illustrating an example network system incommunication with user devices and provider devices, in accordance withexamples described herein. Network system 100 can implement or manage anetwork-based service (e.g., an on-demand transport service, anon-demand delivery service, etc.) that connects requesting users 182with service providers 192 that are available to fulfill the users'service requests 183. The network system 100 can provide a platform thatenables on-demand services to be provided by an available serviceprovider 192 for a requesting user 182 by way of a user application 181executing on the user devices 180, and a provider application 191executing on the provider devices 190. As used herein, a user device 180and a provider device 190 can comprise a computing device withfunctionality to execute a designated application corresponding to theon-demand service managed by the network system 100. In many examples,the user device 180 and the provider device 190 can comprise mobilecomputing devices, such as smartphones, tablet computers, VR or ARheadsets, on-board computing systems of vehicles, smart watches, and thelike. In one example, a service provider fulfilling a service requestincludes the service provider rendezvousing with the user at startlocation (e.g., a pick-up location) to pick up the user and transportingthe user to a service location (e.g., a destination location).

The network system 100 can include a user device interface 115 tocommunicate with user devices 180 over one or more networks 170 via theuser application 181. According to examples, a requesting user 182wishing to utilize the network-based service can launch the userapplication 181 and can cause user device 180 to transmit, by using theuser application 181, a service request 183 over the network 170 to thenetwork system 100. In certain implementations, the requesting user 182can view multiple different service types managed by the network system100. In the context of an on-demand transport service, service types caninclude a ride-share service, an economy service, a luxury service, aprofessional service provider service (e.g., where the service provideris certified), a self-driving vehicle service, and the like. In certainimplementations, the available service types can include arideshare-pooling service class in which multiple users can be matchedto be serviced by a service provider. The user application 181 canenable the user 182 to scroll through the available service types. Theuser application 181 can also enable the user 182 to enter the start andservice locations for a prospective service request. In some examples,the user device 180 can automatically determine the start location basedon the current location of the user 182 (e.g., as determined by on-boardlocation-aware resources).

According to embodiments, the network system 100 can include a serviceengine 125 that can perform a number of functions in response toreceiving the service request 183 from the user device 180. Forinstance, in response to receiving the service request 183, the serviceengine 125 can identify a candidate service provider 192 to fulfill theservice request 183. The service engine 125 can receive providerlocation data 195 transmitted from the provider devices 190 to identifyan optimal service provider 192 to service the user's service request183. The optimal service provider 192 can be identified based on theservice provider 192's location, ETA to the start location, status,availability, and the like.

In various aspects, the service engine 125 can transmit an invitation126 to the provider device 190 of the selected service provider 192. Theinvitation 126 can be transmitted over the network 170 via a providerdevice interface 120 that communicates with provider devices 190. Inresponse to receiving the invitation 126, the provider application 191can display a prompt for the service provider 192 to accept or declinethe invitation 126. Should the service provider 192 accept theinvitation 126, the provider application 191 can cause the providerdevice 190 to transmit an acceptance 193 to the network system 100. Inresponse to receiving the acceptance 193 from the provider device 190,the network system 100 and the service engine 125 can perform a numberof operations to facilitate the fulfillment of the requested service bythe service provider 192. As an example, the service engine 125 generatean optimal route 127 for the service provider 192 to fulfilling theservice request 183. The route 127 can be generated based on map data(e.g., stored in database 145 or from a third-party mapping resource).The route 127 can include a segment from the current location of theservice provider 192 (e.g., based on the provider location data 195) tothe start location and another segment from the start location to theservice location. The route 127 can also include other intermediatelocations such as a drop-off location for another user of a ride-sharetransport service, etc. The provider device interface 120 can transmitthe route 127 to the provider device 190 via the one or more networks170. The provider device 190 can display, via the provider application191, turn-by-turn directions for the service provider 192 based on theroute 127 generated by the service engine 125. In some implementations,the service engine 125 can transmit the start and service locations tothe provider device 190 and the provider devices 190 and the providerapplication 191 can generate one or more routes and turn-by-turndirections for the service provider 192 necessary to fulfill the servicerequest 183.

In various examples, the network system 100 can maintain user data forthe requesting user 182 in the database 145 in the form of user profiledata 148. The user profile data 148 can include information relating toservices requested by the user 182 in the past, frequently visitedlocations associated with the network-based service (e.g., homelocation, office address, etc.), and the like. The user profile data 148can also include payment information (e.g., credit/debit cardinformation, etc.) used by the network system 100 to process the user182's payments for the network-based service. In some implementations,the user 182 can enter payment information via the user application 181.For instance, the user 182 can be prompted, either while setting up auser account or profile for the network-based service or beforesubmitting a request for service.

According to embodiments, the network system 100 includes servicesimulation engine 130, model generation engine 135, and serviceoptimization engine 140 to generate and apply machine-learned serviceprovider optimization (MLSPO) model(s) 147 for generating serviceprovider optimizations 141 for the service provider 192. The servicesimulation engine 130 can perform a plurality of simulations of thenetwork-based service using historical service data 146. The historicalservice data 146 can include some or all aspects of the past instancesof the network-based service, such as fare paid, time of request (e.g.,time of day and/or day of week), duration of service, location ofservice provider when invitation was received, time and distancetraveled (e.g., from location of service provider at time of acceptinginvitation to start location and/or from start location to servicelocation), and the like. Based on the historical service data 146, theservice simulation engine 130 can simulate one or more instances of thenetwork-based service for one or more agents 133.

In some examples, the service simulation engine 130 can receive inputparameters 131. The input parameters 131 can be specified by systemadministrator at the time of the simulations or can be predetermined. Inone implementation, one of the input parameters 131 can specify themetric that the process will seek to optimize. For example, expectedfares for the service provider 192 over a given duration of time canspecified as the metric to be optimized. A simulation time duration alsobe specified as one of the input parameters 131. As an example, if 12hours is specified as the simulation time duration, the servicesimulation engine 130 can simulate 12 hours of activity for the agent133. The resulting MLSPO model 147 based on the results of suchsimulations will generate service provider optimizations 141 thatoptimize a service metric (e.g., expected fares) over a 12-hourduration. In comparison, if 2 hours is specified as the simulationduration, the service simulation engine 130 can simulate 2 hours ofactivity for the agent 133 and the resulting MLSPO model 147 willoptimize the service metric over a 2-hour duration.

According to embodiments, the service simulation engine 130 generatesoutput parameters 132 in simulating the network-based service for theone or more agents 133. The model generation engine 135 can receive theoutput parameters 132 to generate MLSPO model 147. In some examples, themodel generation engine can generate or train the MLSPO model 147 usingreinforcement learning techniques (e.g., Q-learning, Markov decisionprocesses, policy learning, etc.) applied to optimize (e.g., maximize orminimize) one or more of the parameters 132 generated during thesimulations. Once the MLSPO model 147 is generated or trained, it can bestored within database 145 for retrieval in response to an optimizationrequest (e.g., optimization request 194) received from the providerdevice 190.

The network system 100 can receive an optimization request 194 from theprovider device 190 of the service provider 192. The optimizationrequest 194 can include location data that indicates the currentlocation of the service provider 192. For example, after serviceprovider 192 completes fulfilling a service request, the serviceprovider 192 can provide an indication that the requested service hasbeen completed. In some examples, that indication can be theoptimization request 194. In other examples, the service provider 192can interact with the provider application 191 to cause the providerdevice 190 to transmit the optimization request 194.

According to embodiments, the network system 100 can retrieve anappropriate MLSPO model 147 from the database 145 based on theoptimization request 194. The MLSPO model 147 can receive, as an input,the current location of the service provider 192 and the output of theMLSPO model 147 can be service provider optimizations 141. The serviceprovider optimizations 141 can include a route direction for the serviceprovider 192 and one or more action suggestions for the service provider192, both to optimize one or more estimated service metrics (e.g.,estimated or expected fares over a period of time). The actionsuggestions can include entering/exiting an offline mode with respect tothe network-based service, declining or accepting invitations from thenetwork system 100 to fulfill service requests, and the like.

The service provider optimizations 141 can be transmitted to theprovider device 190 via the provider device interface 120 and thenetwork 170. In response to receiving the service provider optimizations141, the provider application 191 executing on the provider device 190can cause information relating to the service provider optimizations 141to be displayed by the provider device 190 to the service provider 192.

Methodology

FIG. 2 is a flow chart illustrating an example method of performingservice provider optimizations, in accordance with examples describedherein. In the below description of FIG. 2, references may be made withrespect to FIG. 1. For instance, the example method illustrated in FIG.2 can be performed by exemplary network system 100 and/or providerdevice 190 illustrated in and described with respect to FIG. 1.

Referring to FIG. 2, the network system can receive model generationparameters (210). The model generation parameters can specify one ormore aspects of the simulations of the network-based service and/or themachine-learning process in generating the machine-learned serviceprovider (MLSPO) models. The model-generation parameters can bepredetermined or can be specified by a system administrator at the timeof generating the MLSPO models or performing the simulations of thenetwork-based service.

One model-generation parameter can be the specification of a servicemetric to be optimized for service providers (211). In someimplementations, the service metric to be optimized can be the totalfare expected to be received by the service provider for providing thenetwork-based service. In other implementations, the MLSPO model can begenerated to optimize distance traveled by the service provider or timewaiting for service requests.

Another model-generation parameter can be a time duration over which toperform simulations and over which the optimizations will be performed(212). As an example, if 12 hours is specified as the time durationparameter, the network system can simulate 12 hours of activity for oneor more agents in performing as service providers of the network-basedservice. The resulting MLSPO model based on the results of suchsimulations will generate service provider optimizations that optimizethe specified service metric (e.g., expected fares) over a 12-hourduration. In comparison, if 2 hours is specified as the time durationparameter, the service simulation engine 130 can simulate 2 hours ofactivity for the one or more agents and the resulting MLSPO model willgenerate service provider optimizations that optimize the service metricover a 2-hour duration.

Other model-generation parameters can also be specified or predetermined(213). In one example, the geographic region over which thenetwork-based service is managed by the network system can be dividedinto a plurality of geographic subdivisions. The geographic subdivisionscan be hexagons or other shapes. The sizes of the geographicsubdivisions can be another model-generation parameter that can be usedto customize and modify the generation of the MLSPO models. As can beappreciated, smaller geographic subdivisions can yield more finely-tunedoptimizations but can also increase the computational power needed togenerate the MLSPO models.

The network system can generate the MLSPO models based on historicalservice data and the model-generation parameters (215). In generatingthe MLSPO models, the network system can perform a plurality ofsimulations of the network-based service for one or more virtual agents.A virtual agent can represent, within the simulations, a serviceprovider of the network-based service. The simulations can be performedbased on historical service data associated with the network-basedservice. For example, the virtual agent can be simulated to be takingvarious actions such as traveling in a particular direction from theagent's location during the in-between service request time period. Inaddition, the historical service data can be used to simulate theresults of the virtual agent's performance in fulfilling simulatedservice requests. The historical service data can include dataindicating various aspects, such as past instances of the network-basedservice rendered by real-world service providers. For instance, thehistorical service data can also indicate parameters and metrics forthose past instances of the network-based service such as fare paid,time of request (e.g., time of day and/or day of week), duration ofservice, location of service provider when invitation was received, timeand distance traveled (e.g., from location of service provider at timeof accepting invitation to start location and/or from start location toservice location), and the like. Results of the simulations are recordedand used to train the MLSPO model using, for example, reinforcementlearning techniques designed to optimize one or more metrics (e.g., thefare). In some embodiments, a single virtual agent is used in thesimulations to generate a single-agent model (216). In otherimplementations, multiple virtual agents are simulated and a multi-agentmodel can be generated (217).

In various aspects, the network system can periodically update MLSPOmodels based on the most-recent historical service data collected by thenetwork system in managing the network-based service. For instance, thenetwork system can update the MLSPO models every month based on the past3 months or 12 months of service data collected for the network-basedservice. In this manner, the MLSPO models can be updated to reflectchanging usage patterns of the network-based service such thatoptimizations generated for service providers are not out-of-date.

In performing service provider optimization for a given service providerafter the machine-learning models to do so have been generated, thenetwork system can receive an optimization request from the providerdevice of the given service provider (220). The service provider caninteract with the provider application executing on the provider deviceto cause the provider device to transmit the optimization request thenetwork system. The optimization request can indicate the currentlocation of the service provider (e.g., via one or more geo-awareresources of the provider device) and/or the service parameter sought tobe optimized. For example, after fulfilling a service request, theprovider application can present a menu or soft selection feature usingwhich the service provider can cause an optimization request to betransmitted. In doing so, the service provider can be presented with anoption to select among two or more service metrics (e.g., expectedfares, wait times, etc.) to be optimized.

In addition or as an alternative, the network system can automaticallybegin the service provider optimization process based on monitoring thelocation and/or the service progress of the service provider. Forexample, the network system can periodically receive, from the providerdevice, location data indicating the current location of the serviceprovider. In response to determining that the provider device is withina predetermined distance of the service location of a service request,the network system can be triggered to begin the process of determiningservice provider optimizations in anticipation of the service providercompleting the service request. As another example, in response toreceiving a service provider's acceptance an invitation to fulfill aservice request to a service location, the network system can begin togenerate optimizations for the service provider based on the MLSPOmodels and the service location (and/or the estimated arrival time atthe service location). In doing so, the optimizations can be readilytransmitted to the provider device when the service provider completesthe requested service at the service location.

In response to receiving the optimization request, the network systemcan determine one or more real-time parameters of the service provider(225). The one or more real-time parameters can reflect real-timeconditions of the network-based service that may be relevant to theservice provider optimization process. The MLSPO models can receive thereal-time parameters as inputs in outputting the service provideroptimizations.

According to embodiments, the network system can retrieve an appropriatemachine-learning model based on the received optimization request (230).In one aspect, the machine-learning model can be retrieved based on theoptimization service metric sought to be optimized, which can beindicated by the optimization request. For example, in response to afirst optimization request indicating that the service metric sought tobe optimized is the expected fare of the service provider, the networksystem can retrieve a machine learned model generated to optimize theexpected fare. In contrast, in response to a second optimization requestindicating that the service metric sought to be optimized is the waittime of the service provider (e.g., wait times for incoming invitationsfrom the network system), the network system can retrieve machinelearned model generated to optimize that service metric. In someexamples, location-specific machine learned models can be used by thenetwork system to generate service provider optimizations. DifferentMLSPO models can be used depending on the service provider's currentlocation. Accordingly, the appropriate MLSPO model can also be retrievedbased on the current location of the service provider indicated in theoptimization request.

In yet another aspect, the appropriate MLSPO model for generatingoptimizations for the given service provider can be retrieved based onthe real-time parameters determined for the service provider at step225. For instance, the network system can have generated two MLSPOmodels—one based on a single-agent simulation of the network-basedservice and another based on multi-agent simulations of thenetwork-based service. Based on a real-time parameter determined for theservice provider indicating a number of service providers near the givenservice providers, one of the two aforementioned MLSPO models can beretrieved. For example, if the number of service providers within apredetermined distance of the service provider (or within the samegeographic subdivision as the given service provider) is below athreshold value, the MLSPO model generated using single-agentsimulations of the network-based service can be utilized. On the otherhand, if the number of such service providers exceeds the thresholdvalue, the MLSPO model generated using multi-agent simulations of thenetwork-based service can be retrieved.

According to embodiments, the network system can utilize the retrievedmachine learned model to generate service provider optimizations basedon the current location of the service provider and/or the one or moredetermined real-time parameters of the service provider (235). Asdiscussed herein, the current location of the service provider can beindicated by the optimization request or can be determined by monitoringtransmission of location data from the provider device. In oneimplementation, the current location and/or the real-time parameters canbe used as inputs to the retrieved MLSPO model. The output of the MLSPOmodel can include route suggestions (236) and action suggestions (237)for the service provider.

In variations, for the route suggestion, the MLSPO model can generate adirection of travel from the current location that is determined, basedon simulations of the network-based service using historical data, to beoptimal in maximizing or minimizing the specified service parameter. Thedirection of travel can be determined and represented as an angle ofdeviation from a line or plane of reference. For instance, the directionof travel can be determined as an angle (e.g., 30 degrees) from truenorth. The network system and/or the provider application executing onthe provider device can be configured to generate route directions(e.g., turn-by-turn navigation directions) based on the determined angleof travel. For instance, the network system and/or the providerapplication can determine a navigable route that most closely takes theservice provider in the angle of travel based on, for example, map data,the current location of the service provider, traffic data, and thelike. A map representation of the route and/or turn-by-turn directionsfor the route can be displayed on the provider device for the serviceprovider to follow.

In some cases, the route suggestion can be for the service provider toremain at or near his or her current location. For example, this canoccur when the optimization is performed to maximize expected fares andthe service provider is already in a location of historically highdemand for the network-based service. Based on these circumstances, thenetwork system can generate route suggestion that the service providershould remain within the geographic subdivision in which he or she iscurrently located. In response, the network system and/or the providerapplication can indicate to the service provider to remain in place(e.g., within the same geographic subdivision as the one in whichservice provider is currently located). As an alternative, the networksystem and/or the provider application can generate routing directionsto a safe waiting place (e.g., to wait for the next invitation tofulfill a service request) for the service provider within thegeographic subdivision. The safe waiting place can be preselected forthe geographic subdivision based on map data.

According to embodiments, for the action suggestions, the MLSPO modelcan generate suggestions for the service provider such as entering orexiting the offline mode for the network-based service as he or shefollows the routing directions from his or her current location. Thesuggestions can also include declining invitations to fulfill servicerequests, and the like.

According to embodiments, the network system is also configured toperform additional optimizations for certain scenarios such as whenmulti-agent optimization is performed (240). When a MLSPO model based onmulti-agent simulations is used for generated service provideroptimizations, multiple optimizations (e.g., one for each of theplurality of agents in the simulations) can be generated. Thus, thenetwork system can additionally determine (e.g., by randomizing) whichof the multiple optimizations is to be received by the given serviceprovider.

The network system can transmit optimization data to the provider deviceof the given service provider (245). In response to receiving theoptimization data, the provider device can display informationcorresponding to the service provider optimizations generated by theMLSPO models.

FIG. 3A is a flow chart illustrating an example method of simulating aninstance of the network-based service as part of the process forgenerating a machine-learned service provider optimization model, asdescribed herein. FIG. 3B is a diagram illustrating an exemplaryvisualization a simulation of an instance of the network-based serviceas part of the process for generating a machine-learned service provideroptimization model, in accordance with examples described herein.Furthermore, in the below discussions of FIG. 3A and FIG. 3B, referencesmay be made with respect to FIG. 1 and/or FIG. 2. For instance, theexample method illustrated in FIG. 3A can be performed by exemplarynetwork system 100 and/or provider device 190 illustrated in anddescribed with respect to FIG. 1. The example method illustrated in FIG.3A can also be performed a part of the step to generate MLSPO models(step 215) illustrated in and described with respect to FIG. 2.

Referring to FIG. 3A, the process 300 include exemplary steps tosimulate an instance of the network-based service using a virtual agent.The virtual agent can be, for purposes of the simulations and thegeneration of a machine-learned service provider optimization (MLSPO)model, a virtual representation of a service provider taking actions ininteracting with the network-based service to fulfill service requests.The virtual agent can be instantiated by the network system prior torunning a plurality of simulations to generate the MLSPO model. Thenetwork system can simulate various actions performed by the virtualagent and can also simulate the results (e.g., parameters such asdistance and time traveled, fare received) of those actions performed.The simulated actions and results of one or more virtual agents arerecorded and used to train (e.g., using reinforcement learning) one ormore MLSPO models in attempting to optimize one or more service metrics(e.g., maximizing expected fares, minimizing wait times, minimizingtravel distances, etc.).

According to embodiments, the geographic region over which the networksystem manages the network-based service can comprise a plurality ofgeographic subdivisions. In one example, as illustrated in FIG. 3B, thegeographic subdivisions can be hexagons. These geographic subdivisionscan be used for the provision of the network-based service in the realworld. For instance, dynamic determinations of metrics such as fares fora given geographic subdivision can be dependent on the presence andnumber of available service providers within the given geographicsubdivision or neighboring geographic subdivisions. The sizes of thegeographic subdivisions can be predetermined based on the variousfactors such as typical demand for the network-based service within eachof the subdivisions.

The network system can determine an initial location (S) for theinstance of the network-based service to be simulated (310). Referringto FIG. 3B, for example, to simulate for and model the behavior avirtual agent starting within subdivision 350, the network system canbegin simulations for the virtual agent at subdivision 350. The processwill simulate the actions and results of the virtual agent as the agentprogresses from subdivision to subdivision within the geographic regionin fulfilling the network-based service. In the example illustrated inFIG. 3B, the virtual agent is simulated to progress from subdivision 350to subdivision 360 to subdivision 370 and finally to subdivision 380. Asimulation of the next instance (not illustrated in FIG. 3B) of thevirtual agent can thus begin at subdivision 380.

According to embodiments, simulating an instance of the network-basedservice includes simulating actions and results of the virtual agentduring an off-service segment, a transit segment, and an on-servicesegment. The off-service segment can refer to the time period betweencompleting a prior service request and accepting an invitation tofulfill a service request. The transit segment can refer to the timeperiod between accepting the invitation and arrival at the startlocation of the service request. And the on-service segment can refer tothe time period during which the virtual agent is simulated to betraveling from the start location to the service location in fulfillmentof the service request.

The network system can simulate the off-service segment of thenetwork-based service (320). Referring to FIG. 3B, the off-servicesegment can be segment 365 from the initial location (S) in subdivision350 to a location (A) in subdivision 360. The network system can alsosimulate a transit segment, which can be a segment 375 from the location(A) in subdivision 360 to a location (B) in subdivision 370. The segment365 can represent the actions of the virtual agent before the agentaccepts a (virtual) invitation to fulfill a service request. The segment375 can represent the actions of the virtual agent as the agent proceedsto a start location (location (B)) of the service request. The networksystem can also simulate on-service segment 385 (330) from startlocation (B) located in subdivision 370 to the service location (S′)located in subdivision 380.

In some embodiments, one or more of the segments 365, 375, and/or 385may not be individually or explicitly simulated. Rather, in addition oras an alternative, the approach taken to simulate the virtual agentactions during the off-service segment can be done by simulating thevirtual agent traveling in a given direction from the initial location(S). The given direction of travel can be represented as an angle 355from a reference direction or reference line. In some examples, thenetwork system can perform a general statistical modeling, based onhistorical service data, of past actions and results of the serviceproviders traveling in a given direction rather than performing detailedsimulations of the individual segments 365, 375, and/or 385. In onevariation, the network system can model the virtual agent traveling fromthe initial location (S) in a direction represented by angle 355 andsimulating the parameters associated with the network-based service inthe virtual agent arriving at the service location (S′). In many cases,particularly where the geographical region comprises a great number ofsubdivisions, this can result in lower demand on computational power toperform the simulations. Furthermore, the network system can beconfigured to simulate virtual agent actions for a plurality of anglesof travel (e.g., every 20°) in order to obtain a complete set ofsimulations for the virtual agent starting from initial location (S).For instance, the network system can perform a first set of simulationsfor the virtual agent with an angle of travel of 0° from the initialposition S, a second set of simulations for the virtual agent with anangle of travel of 20°, a third set of simulations for the virtual agentwith an angle of 40°, etc. In this manner, the network system cansimulate a complete set of possible routes and actions of the serviceprovider in determining one or more directions of travel and/or actionsto optimize the one or more service metrics.

The simulation can be performed based on historical service datamaintained by the network system. In one example, the network system canperform statistical modeling based on the historical service data toobtain results (e.g., by computing service parameters) of the virtualagent performing the on-service segment. Computed service parameterssuch as fare received, time elapsed, distance traveled, etc. can becomputed and used to train the MLSPO model (340). After performing thesimulation of the virtual agent's travel from initial location (S) tothe service location (S′), the network system can continue performingadditional simulations for the virtual agent. For instance, the networksystem can perform an additional simulation of the virtual agent'sactions from S′ to the next service location (S″, not illustrated inFIG. 3B).

FIG. 4 is a diagram illustrating an exemplary visualization of serviceprovider optimizations provided by an exemplary machine-learned serviceprovider optimization (MLSPO) model, in accordance with examplesdescribed herein. As in FIG. 3B, the geographic region can comprise aplurality of subdivisions such as subdivision 410 and 420.

For subdivision 410, the MLSPO model can generate route direction oftravel to optimize one or more service metrics for the service provider.Thus, if the service provider is determined to be located within thesubdivision 410, and service provider optimization is requested, thenetwork system can generate a routing direction 415 for the serviceprovider. In some examples, the routing direction 415 outputted by theMLSPO model can be a direction of travel represented as an angle 416from reference direction (e.g., cardinal north). The network systemand/or the provider application executing on the provider device can beconfigured to generate route directions (e.g., turn-by-turn navigationdirections) based on the routing direction 415 and/or the angle 416. Forinstance, the network system and/or the provider application candetermine a navigable route that most closely takes the service providerthat conforms to the angle 416 based on, for example, map data, thecurrent location of the service provider, traffic data, and the like. Amap representation of the route and/or turn-by-turn directions for theroute can be displayed on the provider device for the service providerto follow.

In contrast, for subdivision 420, the output of the MLSPO model can be adirection for the service provider to remain in place 425 (or within thesubdivision 420). At a high level, this can occur, for example, when theservice provider requests to optimize expected fares and the serviceprovider is already located in an area (e.g., subdivision 420) of highdemand for the network-based service. Thus, to optimize expected fares,the MLSPO model may generate a route direction informing the serviceprovider to remain in place.

Hardware Diagrams

FIG. 5 is a block diagram illustrating an example mobile computingdevice, in accordance with examples described herein. In manyimplementations, the mobile computing device 500 can be a smartphone,tablet computer, laptop computer, VR or AR headset device, and the like.In the context of FIG. 1, the user device 180 and/or the provider device190 may be implemented using a mobile computing device 500 asillustrated in and described with respect to FIG. 4.

According to embodiments, the mobile computing device 500 can includetypical telephony features such as a microphone 570, a camera 540, and acommunication interface 510 to communicate with external entities (e.g.,network system 590 implementing or managing the network-based service)using any number of wireless communication protocols. The mobilecomputing device 500 can store a designated application (e.g., a serviceapplication 532) in a local memory 530. The service application 532 cancorrespond to one or more user applications for implementations of themobile computing device 500 as user devices for the network-basedservice. The service application 532 can also correspond to one or moreprovider applications for implementations of the mobile computing device500 as provider devices for the network-based service.

In response to an input 518, the processor can execute the serviceapplication 532, which can cause an application interface 542 to begenerated on a display screen 520 of the mobile computing device 500. Inimplementations of the mobile computing device 500 as user devices, theapplication interface 542 can enable a user to, for example, request forthe network-based service. The request for service can be transmitted tothe network system 590 as an outgoing service message 567.

In various examples, the mobile computing device 500 can include a GPSmodule 550, which can provide location data 562 indicating the currentlocation of the mobile computing device 500 to the network system 590over a network 580. In some implementations, other location-aware orgeolocation resources such as GLONASS, Galileo, or BeiDou can be usedinstead of or in addition to the GPS module 560. The location data 562can be used in generating a service request, in the context of themobile computing device 500 operating as a user device. For instance,the user application can set the current location as indicated by thelocation data 562 as the default start location (e.g., a location wherea selected service provider is to rendezvous with the user).

FIG. 6 is a block diagram that illustrates a computer system upon whichexamples described herein may be implemented. A computer system 600 canrepresent, for example, hardware for a server or combination of serversthat may be implemented as part of a network service for providingon-demand services. In the context of FIG. 1, the network system 100 maybe implemented using a computer system 600 or combination of multiplecomputer systems 600 as described by FIG. 6.

In one aspect, the computer system 600 includes processing resources(e.g., processor 610), a main memory 620, a memory 630, a storage device640, and a communication interface 650. The computer system 600 includesat least one processor 610 for processing information stored in the mainmemory 620, such as provided by a random access memory (RAM) or otherdynamic storage device, for storing information and instructions whichare executable by the processor 610. The main memory 620 also may beused for storing temporary variables or other intermediate informationduring execution of instructions to be executed by the processor 610.The computer system 600 may also include the memory 630 or other staticstorage device for storing static information and instructions for theprocessor 610. A storage device 640, such as a magnetic disk or opticaldisk, is provided for storing information and instructions.

The communication interface 650 enables the computer system 600 tocommunicate with one or more networks 680 (e.g., a cellular network)through use of a network link (wireless or wired). Using the networklink, the computer system 600 can communicate with one or more computingdevices, one or more servers, and/or one or more self-driving vehicles.In accordance with some examples, the computer system 600 receivesservice requests from mobile computing devices of individual users. Theexecutable instructions stored in the memory 630 can include providerselection instructions 622, machine-learned model generationinstructions 624, and service provider optimization 626 to perform oneor more of the methods described herein when executed.

By way of example, the instructions and data stored in the memory 620can be executed by the processor 610 to implement an example networksystem 100 of FIG. 1. In performing the operations, the processor 610can handle service requests and provider statuses and submit serviceinvitations to facilitate fulfilling the service requests. The processor610 executes instructions for the software and/or other logic to performone or more processes, steps and other functions described withimplementations, such as described herein with respect to FIGS. 1-4.

Examples described herein are related to the use of the computer system600 for implementing the techniques described herein. According to oneexample, those techniques are performed by the computer system 600 inresponse to the processor 610 executing one or more sequences of one ormore instructions contained in the main memory 620. Such instructionsmay be read into the main memory 620 from another machine-readablemedium, such as the storage device 640. Execution of the sequences ofinstructions contained in the main memory 620 causes the processor 610to perform the process steps described herein. In alternativeimplementations, hard-wired circuitry may be used in place of or incombination with software instructions to implement examples describedherein. Thus, the examples described are not limited to any specificcombination of hardware circuitry and software.

It is contemplated for examples described herein to extend to individualelements and concepts described herein, independently of other concepts,ideas or systems, as well as for examples to include combinations ofelements recited anywhere in this application. Although examples aredescribed in detail herein with reference to the accompanying drawings,it is to be understood that the concepts are not limited to thoseprecise examples. As such, many modifications and variations will beapparent to practitioners skilled in this art. Accordingly, it isintended that the scope of the concepts be defined by the followingclaims and their equivalents. Furthermore, it is contemplated that aparticular feature described either individually or as part of anexample can be combined with other individually described features, orparts of other examples, even if the other features and examples make nomentioned of the particular feature. Thus, the absence of describingcombinations should not preclude claiming rights to such combinations.

What is claimed is:
 1. A network system for managing a network-basedservice, comprising: one or more processors; and one or more memoryresources storing instructions that, when executed by the one or moreprocessors of the network system, cause the network system to: generate,based on a set of historical data associated with the network-basedservice, a machine-learned optimization model for generating actionrecommendations for service providers of the network-based service;determine, based on a current location of a service provider and themachine-learned optimization model, one or more action recommendationsfor optimizing one or more estimated service metrics for the serviceprovider in fulfilling one or more service requests of the network-basedservice; transmit, to a provider device of the service provider, a setof data to cause the provider device to display information regardingthe determined one or more action recommendations for the serviceprovider; and wherein the one or more action recommendations includes:(i) a recommended direction of travel for the service provider while theservice provider is traveling on an off-service segment, and (ii) arecommendation for the service provider to enter an offline state withrespect to the network-based service.
 2. The network system of claim 1,wherein the recommended direction of travel is defined as an angle oftravel with respect to a reference direction; wherein the executedinstructions further cause the network system to generate, based on theangle of travel with respect to the reference direction and the currentlocation of the service provider, navigation instructions for theservice provider; and wherein the information displayed on the providerdevice regarding the determined one or more action recommendations forthe service provider correspond to the navigation instructions.
 3. Thenetwork system of claim 1, wherein the one or more actionrecommendations further includes a recommendation for the serviceprovider to travel to a pre-determined waiting area for waiting for aninvitation to fulfill a service request.
 4. The network system of claim1, wherein generating the machine-learned optimization model furtherincludes: performing, for a first virtual agent based on the set ofhistorical data, a first set of agent-based simulations of thenetwork-based service including a first agent-based simulation from afirst location to a second location; and determining, for each one thefirst set of agent-based simulations of the network-based service, acorresponding value of a first service metric of the one or moreestimated service metrics based on the set of historical data.
 5. Thenetwork system of claim 4, wherein the machine-learned optimizationmodel is trained based on results of the first set of agent-basedsimulations to maximize or minimize the first service metric.
 6. Thenetwork system of claim 4, wherein generating the machine-learnedoptimization model further includes: performing, for a second virtualagent based on the set of historical data, a second set of agent-basedsimulations of the network-based service.
 7. The network system of claim1, wherein the current location of the service provider is provided asan input to the machine-learned optimization model to determine the oneor more action recommendations for optimizing one or more estimatedservice metrics for the service provider in fulfilling one or moreservice requests of the network-based service.
 8. The network system ofclaim 1, wherein the executed instructions further cause the networksystem to: determine a real-time parameter based on one or morereal-time conditions of the network-based service; and wherein thereal-time parameter as an input to the machine-learned optimizationmodel to determine the one or more action recommendations for optimizingone or more estimated service metrics for the service provider infulfilling one or more service requests of the network-based service. 9.The network system of claim 8, wherein the real-time parameter is aparameter indicating a number of other available service providerswithin a predetermined distance of the service provider.
 10. The networksystem of claim 1, wherein the set of historical data indicatesrespective values of a service parameter associated with each of aplurality of past instances of the network-based service.
 11. Thenetwork system of claim 1, wherein the one or more estimated servicemetrics includes one or more of the following service metrics for theservice provider in fulfilling requests for the network-based serviceover a future period of time: (i) expected fares, (ii) expected waittimes, or (iii) expected travel distances.
 12. The network system ofclaim 11, wherein the future period of time is specified by the serviceprovider and wherein the one or more action recommendations isdetermined based further on the future period of time.
 13. The networksystem of claim 1, wherein the machine-learned optimization model isgenerated using reinforcement learning.
 14. A computer-implementedmethod comprising: generating, based on a set of historical dataassociated with a network-based service, a machine-learned optimizationmodel for generating action recommendations for service providers of thenetwork-based service; determining, based on a current location of aservice provider and the machine-learned optimization model, one or moreaction recommendations for optimizing one or more estimated servicemetrics for the service provider in fulfilling one or more servicerequests of the network-based service; transmitting, to a providerdevice of the service provider, a set of data to cause the providerdevice to display information regarding the determined one or moreaction recommendations for the service provider; and wherein the one ormore action recommendations includes: (i) a recommended direction oftravel for the service provider while the service provider is travelingon an off-service segment, and (ii) a recommendation for the serviceprovider to enter an offline state with respect to the network-basedservice.
 15. The computer-implemented method of claim 14, whereingenerating, based on the set of historical data associated with thenetwork-based service, the machine-learned optimization model forgenerating action recommendations for service providers of thenetwork-based service comprises: performing, for a first virtual agentbased on the set of historical data, a first set of agent-basedsimulations of the network-based service including a first agent-basedsimulation from a first location to a second location; and determining,for each one the first set of agent-based simulations of thenetwork-based service, a corresponding value of a first service metricof the one or more expected service metrics based on the set ofhistorical data; and wherein the machine-learned optimization model istrained based on results of the first set of agent-based simulations tomaximize or minimize the first service metric.
 16. A non-transitorycomputer-readable medium storing instructions that, when executed by oneor more processors of a network system, cause the network system to:generate, based on a set of historical data associated with anetwork-based service, a machine-learned optimization model forgenerating action recommendations for service providers of thenetwork-based service; determine, based on a current location of aservice provider and the machine-learned optimization model, one or moreaction recommendations for optimizing one or more estimated servicemetrics for the service provider in fulfilling one or more servicerequests of the network-based service; transmit, to a provider device ofthe service provider, a set of data to cause the provider device todisplay information regarding the determined one or more actionrecommendations for the service provider; and wherein the one or moreaction recommendations includes: (i) a recommended direction of travelfor the service provider while the service provider is traveling on anoff-service segment, and (ii) a recommendation for the service providerto enter an offline state with respect to the network-based service. 17.The non-transitory computer-readable medium of claim 16, whereingenerating, based on the set of historical data associated with thenetwork-based service, the machine-learned optimization model forgenerating action recommendations for service providers of thenetwork-based service comprises: performing, for a first virtual agentbased on the set of historical data, a first set of agent-basedsimulations of the network-based service including a first agent-basedsimulation from a first location to a second location; and determining,for each one the first set of agent-based simulations of thenetwork-based service, a corresponding value of a first service metricof the one or more expected service metrics based on the set ofhistorical data; and wherein the machine-learned optimization model istrained based on results of the first set of agent-based simulations tomaximize or minimize the first service metric.
 18. Thecomputer-implemented method of claim 14, wherein the one or more actionrecommendations further includes a recommendation for the serviceprovider to travel to a pre-determined waiting area for waiting for aninvitation to fulfill a service request.
 19. The computer-implementedmethod of claim 15, wherein the machine-learned optimization model istrained based on results of the first set of agent-based simulations tomaximize or minimize the first service metric.
 20. Thecomputer-implemented method of claim 15, wherein generating themachine-learned optimization model further includes: performing, for asecond virtual agent based on the set of historical data, a second setof agent-based simulations of the network-based service.